This invention relates to machine vision, and particularly to dimensional measurement within a digital image.
Many machine vision systems use a model for inspecting or recognizing objects within images. Often, such models must accurately reflect the physical dimensions of the objects so that the position of the objects can be determined in an image, and so that precise tolerance checking and defect detection can be performed. For example, a vision-based automatic surface mounter (SMD) machine employs a model that includes the length of leads of leaded devices to accurately inspect and place the leaded devices on a printed circuit board (PCB).
FIG. 1A illustrates a bottom view and a side view of a gullwing-leaded device 100, not drawn to scale, where a leaded device is an electronic device that has a device body 110 and leads 102. The leads 102 are metal contacts on the exterior surface of the device body 110 that are connected to an integrated circuit (not shown) within the device body 110. Leaded devices include surface-mount devices and through-hole devices, for example. The leads 102 of the surface-mount devices are placed by the SMD machine, such that the leads 102 substantially contacts pads on a PCB (not shown) within positional tolerances.
The length 104 of a gullwing lead 102 is the distance between the base 108 and the tip 106 of the lead 102. The positions of bases 108 and tips 106 in an image are often determined by identifying edges corresponding to the bases 108 and the tips 106. With varying degrees of success, the edges are found using methods known in the art. The edges can also be found by using CALIPER TOOL sold by Cognex Corporation. The CALIPER TOOL is a machine-vision tool illustrated with reference to FIG. 2 and further described in Vision Tools, Chapter 4, CALIPER TOOL, Cognex Corporation, Version 7.4, 1996, pp. 208-231, incorporated herein by reference.
The CALIPER TOOL finds edges, such as 206 and 208, within an area of an image 200 enclosed by a window 204. More particularly, the CALIPER TOOL accumulates edges along a projection axis, p, of the window 204. An edge, as used herein, consists of a plurality of connected edge elements or edge pixels that correspond to underlying image pixels. An image can be represented as an array of image pixels, where an image pixel is a picture element characterized by a grey value. Each edge can be one or more image pixels wide.
The intensity of pixels within the window 204 along p are projected (i.e., added), thereby generating a one-dimensional image 210. The projection axis, p, is perpendicular to l, and together l, p, and w, which is the width of the window 204, define the window 204. Linear projection collapses an image by summing the grey values of the pixels in the direction of the projection. The summation tends to amplify edges in the same direction as p. After projection, an edge filter is applied to the one-dimensional image 210 to further enhance edge information and to smooth the one-dimensional image 210. The one-dimensional image is illustrated graphically as histogram 212. The edge 206 is represented in the histogram 212 as a falling ramp 216, and the edge 208 is represented in the histogram as a rising ramp 218. Each edge 206 and 208 has a polarity (i.e., direction), where edge 206 has a light-to-dark polarity and edge 208 has a dark-to-light polarity. In this example, both edges 206 and 208 have the same contrast, where contrast is the difference in grey levels on opposite sides of an edge.
The edges corresponding to the bases 108 and the tips 106 of the leads 102 are located using a window of the CALIPER TOOL. To find the edges corresponding to the bases 108 and the tips 106, optimally projection is performed along a direction of a line tangent to the edges of the bases 108 and the tips 106. Accordingly, the window is positioned such that its projection axis, p, is as close as possible substantially parallel to a line tangent to the lead bases 108 and the lead tips 106 and normal to a lead axis, T-Txe2x80x2, of the leads 102. The degree p can be of offset from parallel depends upon each application and varies widely as is known in the art. The length, l, extends across each end of the leads 102. Alternatively, two windows 112 and 114 each enclosing one end of each lead can locate each base 108 and tip 106, where p of the two windows is also substantially parallel to a line tangent to the bases 108 and tips 106 and substantially normal to T-Txe2x80x2.
A problem with these methods is the lack of integrity of the generated edge information. Other structures in the image generate extraneous edges, such as feature 302 on the device body 300 and the silhouette of the device body 304, not drawn to scale, illustrated in gullwing-leaded device of FIG. 3. The extraneous edges confuse identification of the edges of the lead base 308 and lead tip 306.
A further drawback of this method is that back-lit images do not have edges corresponding to both the lead base 308 and the lead tip 306. FIG. 4B illustrates back-lit imagery, where the light 450 originates from a light source 458 located behind the object 452 and is directed toward the imaging device 456 so that the object 452 and the leads 454 appear as a silhouette.
Another drawback of this method is that it is not easily extended to deal with varying lead lengths.
Alternatively, the length of leads is determined by binarizing the image of the leaded device. Binarizing an image is a technique where a threshold is chosen to segment the image into foreground objects and the background. Typically, one intensity, such as white, denotes the leads, and the other intensity, such as black, denotes the image background. Once binarized, the length of the white object in the image is determined easily using known methods, such as a connected component analysis.
One of the short falls of the binarization technique is the inability of a single threshold to segment the entire lead from the background. Typically, the leads have specularly reflecting surfaces that frustrate identifying a threshold within a front-lit image of a leaded device. FIG. 4A illustrates a front-lit system, where a light source 408 directs light 410 towards a bottom of a leaded device 402 and leads 404, and the light reflects off the leaded device 402 and the leads 404 back to the imaging device 406 which collects the light. The metal leads 404 specularly reflect the light 410 of the front-lit system. Further, the shape of the leads 404 causes reflections in some portions of the leads 404 to be stronger than other reflections. An example of the varying intensities imaged from a portion of a gullwing-leaded device is illustrated in FIG. 3.
Back-lit images of the leaded devices do not exhibit specular reflections because only the silhouette of the device is imaged. In a back-lit image, the leads and the device body have substantially the same grey scale value, and, therefore, no threshold exists that segments the entire lead relative to the body and background. Consequently, the base of the leads cannot be identified in the image. Thus, the binarization method is not an optimal solution.
In addition to leads, other parallel objects that are in close proximity to each other often frustrate prior methods for measuring length of the parallel objects.
Methods and apparatuses are disclosed for measuring an extent of a group of objects within a digital image by comparing a reference signature, representative of at least one relationship of the objects to one another, against instances of a measured signature representing various positions within the image. The position(s) where the signature(s) vary by more than a predetermined comparison criteria are used to calculate the extent of the group of objects. More generally, the comparison criterion indicates when the measured signature no longer represents the same group of objects.
The measured signature is obtained by placing a window having a projection axis substantially parallel to the extent of the group of objects, where the window extends across at least a portion of the group of objects. The image within the window is projected along the projection axis of the window to generate a measured signature that is then compared against the reference signature.
A measured signature is generated at a plurality of window positions along the extent of the group of objects. The measured signature is generated until the measured signature differs from the reference signature by more than the comparison criteria. When the measured signature differs by more than the comparison criteria, the group of objects is considered no longer within the window. Therefore, the group of objects has an extent that is no greater than the position of the corresponding measured signature. Consequently, the position of the measured signature where the measured signature differs by the comparison criteria from the reference signature is used to calculate the extent of the group of objects.
In one embodiment, the extent is approximately the difference between two positions of the measured signature just prior to the measured signatures that indicate the objects are no longer within the window. In this embodiment, the image is searched by moving the windows outward from one center point until both ends of the group of objects are found.
In one embodiment, the reference signature is a signature of the group of objects near one of its ends, but not including the ends. Therefore, the extent of the group is approximately the difference between the window position of the reference signature and the window position of the measured signature just prior to the measured signature that indicates the group of objects is no longer within the window.
In one embodiment, the measurement of the extent of the objects is further refined by examining at a finer increment the space between the two consecutive positions of the measured signatures that indicated the termination of the group of objects. Specifically, multiple measured signatures are generated between the two consecutive positions of the window. Therefore, more precisely determining the position of the termination of the group of objects.
In a preferred embodiment, the signatures are generated using a window of the CALIPER TOOL, and the signatures are related to edges of the group of objects. In a preferred embodiment, the measured signature is a collection of the edges within the window, and the reference signature has slots derived from edges. Each slot of the reference signature represents an edge position plus or minus a positional tolerance, where each slot is xe2x80x9cfilledxe2x80x9d by a correspondingly positioned edge of the measured signature.
In further aspects, the reference signature is modified to accommodate expected gradual movement of the objects from their original position in the reference signature.
A preferred application is measuring the length of leads in a leaded device, but other non-parallel object applications are also disclosed.
Because the reference signature is predetermined, the comparison of positions in the image to the reference signature accommodates a wide degree of noise and other randomness within the image without altering the accuracy of the measurement of the extent of the group of objects.
The invention recognizes that focusing only on position of the edges within the measured signature also provides latitude for noise, while still properly determining the extent of the objects. Further, one advantage of the invention is that it can measure objects that are in close proximity to each other.
The method and apparatuses according to the invention overcome various problems with prior art measuring methods, such as problems resulting from extraneous edges, problems resulting from varying lengths of groups of objects, and problems resulting from difficulties in segmenting the objects from the remainder of the image, for instance.